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Abstract of EP0969387 

A moving picture editing method and apparatus that edits a moving picture fed from a video material 
source by working on editing windows displayed on a display (208) to divide a video material into a 
plurality of cut images each formed of a plurality of frame images (310 SIMILAR 317). display the 
representative frame images of each of said cut images on the screen of the display, select the 
representative frame images, assemble a hierarchical structure (802) formed of a plurality of frame 
images, display the hierarchical structure, select one or a plurality of the representative frame images as a 
scene image, incorporate the frame images representative of the scene image in the hierarchical 
structure, display a tentative indicator image capable of representing an arbitrary video material on the 
display, incorporate said tentative indicator image in the hierarchical structure, and thus generate the 
hierarchical structure. 
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Editing method and apparatus for moving pictures 

Description of EP0969387 ; ^Translate this text \ 



BACKGROUND OF THE INVENTION 



[0001] The present invention generally relates to moving picture editing methods and apparatus for use in 
producing nhoving picture programs such as television broadcast programs and video programs or in other 
video edition for business or consumers, and particularly to an editing method and apparatus for editing 
moving pictures by selecting icons of representing image materials that are displayed for editing on 
windows of a display screen under the control of a computer using necessary program software, 

[0002] Recently, in cooperation with high-speed, multi-function computers, the computer software has been 
rapidly developed to be diversified and to have multiple functions. These computers and software products 
are now used in equipment for producing television broadcast programs and video programs, thus 
achieving multifunction edition. 

[0003] Particularly in the recent program production, the edition of image information including image data 
and audio data makes use of a moving picture editing apparatus having an information storage unit capable 
of random access, such as hard disks that are easy to handle and store and inexpensive. Furthermore, the 
multifunctional and diverse computers and software products are employed for that edition. 

[0004] There is a nonlinear editing system as an example of the moving picture editing apparatus. In this 
system, video materials are once stored in a disk-type storage unit capable of random access, and 
representative images of the image materials are read from the storage unit and displayed on the screen of 
the editing apparatus. The operator operates on the display screen to edit moving pictures. In this editing 
operation, it is necessary that moving pictures be previously stored and that the representative images and 
video materials be made to be associated with each other. 

[0005] When a program is produced, there are a case where a program structure is already determined but 
video materials are not yet stored in disks, and a case where video materials are already stored in disks but 
which images are used is not determined. For example, in the production of a regular news program, even 
if the time and structure of the program are determined, the latest news images may not be gathered until 
the start of actual program becomes close. In that case, since the program edition cannot be started before 
the image materials to be used are determined, enough time cannot be consumed for the edition, or only a 
little time is left for edition. In addition, it is impossible to employ the way that after the construction of 
program is determined the image materials suitable for that program structure are gathered to produce the 
program. 



SUMMARY OF THE INVENTION 



[0006] The present invention is to provide a moving picture editing method and apparatus capable of 
solving the above problems, or producing a tree structure of images for a program even if the moving 
pictures to be edited are not stored in an information storage unit. Also, in this method and apparatus, even 
if the image information has been already erased from the storage unit after the end of the moving picture 
edition, the lost image materials can be again stored in the storage unit and edited without again generating 
the tree structure. 

[0007] According to the present invention, there is provided a moving picture editing method and apparatus 
in which a moving picture fed from a video material source is edited by working on editing windows 
indicated on a display to. divide a video material into a plurality of cut images each formed of a plurality of 
frame images, display the representative frame images of each of said cut images on the screen of the 
display, selecting the representative frame images, assembling a hierarchical structure formed of a plurality 
of frame images, displaying the hierarchical structure, selecting one or a plurality of the representative 
frame images as a scene image, incorporating the frame images representative of the scene images in the 
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hierarchical structure, displaying a tentative indicator image capable of representing an arbitrary video 
material, incorporating said tentative indicator image in the hierarchical structure, and thus generating the 
hierarchical structure. 

[0008] According to the invention, there is provided a method of displaying images of editing operations of 
an editing apparatus according to the invention, the method including the steps of displaying a first region 
for a video material and a second region for a hierarchical structure having a plurality of images of the video 
material in the first region on the screen of the display, displaying in the first region the frame images 
representative of a cut image formed of a plurality of frame images, displaying in the second region the 
hierarchical structure formed of a plurality of the representative frame images, and displaying a tentative 
indicator image capable of representing an arbitrary video material at a specified position within the 
hierarchical structure. 

[0009] According to the invention, there is provided a computer program product that includes a computer 
readable medium having recorded thereon a computer readable program, when the program is loaded, to 
make the computer execute the procedure to divide a video material into a plurality of cut images each 
formed of a plurality of frame images, display the representative frame images of each of the cut images on 
the screen of the display, select the representative frame images, assemble a hierarchical structure formed 
of a plurality of frame images, display the hierarchical structure, select one or a plurality of the 
representative frame images as a scene image, incorporate the frame images representative of the scene 
image in the hierarchical structure, generate a tentative indicator image capable of representing an arbitrary 
video material, and incorporate the tentative indicator image in the hierarchical structure, thereby editing a 
moving picture. 

[0010] Also, according to the invention, there is provided a computer readable medium having a computer 
readable program recorded thereon where, when the medium is loaded, the program makes the computer 
execute the procedure to divide a video material into a plurality of cut images each formed of a plurality of 
frame images, display the representative frame images of each of the cut images on the screen of the 
display, select the representative frame images, assemble a hierarchical structure formed of a plurality of 
frame images, display the hierarchical structure, select one or a plurality of the representative frame images 
as a scene image, incorporate the frame images representative of the scene image in the hierarchical 
structure, generate a tentative indicator image capable of representing an arbitrary video material, and 
incorporate the tentative indicator image in the hierarchical structure, thereby editing a moving picture. 



BRIEF DESCRIPTION OF THE DRAWINGS 



Fig. 1 is a diagram to which reference is made in explaining displayed editing images in an embodiment of 
a moving picture editing method according to the invention. 

Fig. 2 is a block diagram of an embodiment of a moving picture editing apparatus according to the 
invention. 

Fig. 3 is a diagram to which reference is made in explaining displayed editing images in the embodiment of 
a moving picture editing method according, to the invention. 

Fig. 4 is a diagram to which reference is made in explaining an editing window used in the embodiment of a 
moving picture editing method according to the invention. 

Fig. 5 is a diagram to which reference is made in explaining displayed editing images in the embodiment of 
a moving picture editing method according to the invention. 

Fig. 6 is a diagram to which reference is. made in explaining displayed editing images in the embodiment of 
a moving picture editing method according to the invention. 

Fig. 7 is a diagram to which reference is made in explaining displayed editing images in the embodiment of 
a moving picture editing method according to the invention. 

Fig. 8 shows one example of the displayed editing images in the moving picture editing apparatus. 
Fig. 9 is a diagram to which reference is made in explaining the structure of one example of a moving 
picture information file. 

Fig. 10 is a diagram to which reference is made in explaining the structure of other examples of a moving 
picture information file. 

Fig. 11 is a diagram to which reference is made in explaining the displayed editing image of a hierarchical 
structure according to the moving picture editing method. 
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Fig. 12 consists of Figs. 12A and 12B, and shows a flowchart of a software program for implementing the 

embodiment of a moving picture editing method according to the invention. 

Figs. 13A and 13B are diagrams for explaining the generation of the hierarchical structure. 



DESCRIPTION OF THE EMBODIMENTS 



[0012] Embodiments of a moving picture editing method and apparatus according to the invention will be 
described with reference to the accompanying drawings. Before the detailed description of the 
embodiments, the construction of the nonlinear editing system and how to edit will be described to which 
the invention can be applied. 

(001 3] Fig. 2 is a block diagram of one example of the construction of a moving picture editing apparatus for 
executing a moving picture editing method according to the invention. 

[0014] This moving picture editing apparatus is constructed to include a central processing unit (CPU) 201 
for variously controlling each element of the apparatus and making the editing processes according to 
necessary program software, a display 208 for displaying control information from CPU 201 and information 
concerning the edition of moving picture, scene image, cut image and the hierarchical structure (tree 
structure) indicating the situation of the edition, an input unit 209 such as mouse and keyboard for selecting 
information indicated on the display 208 and supplying commands to CPU 201, a frame buffer memory 207 
for storing image information displayed on the display 208, an image reproducer (for example, a video 
cassette recorder) 205 for reproducing from a recording medium the video signal and audio signal of 
moving pictures being used for the edition and supplying those signals to the outside, and a video interface 
204 for converting the video signal of moving pictures reproduced from the image reproducer 205 into 
image information of the format to be treated in the moving picture editing apparatus and supplying it to the 
information storage unit 206 or for converting the video information of moving pictures edited in that format 
and produced from the information storage unit 206 into the video signal, and supplying it to the outside 
from a video signal output terminal 21 1 . This apparatus also includes a scene changing point detector 203 
for comparing adjacent frame images of the frame images constituting the moving pictures and deciding 
that when image information as the difference between the frame images changes to exceed a 
predetermined amount the adjacent images belong to a different cut image, a compressor/expander 212 for 
compressing or expanding the image information that undergoes format conversion for use in the apparatus 
and that is produced from the video interface 204, a memory 202 for storing various control program 
software to be used in the CPU 201, and temporarily storing various control information being used in this 
editing apparatus, and an information storage unit 206 capable of random access and using, for example, 
hard disks, and . a bus 210 for the digital information transmission among the various units of the moving 
picture editing apparatus. This information storage unit 206 stores the Image information of moving pictures 
for the edition that are reproduced from the image reproducer 205 and converted into a necessary format 
by the video interface 204, and stores editing information and control information for use in this apparatus 
that are temporarily stored in the memory 202, for example, image information files concerning reduced 
images being displayed. In addition, moving picture data can be supplied to this apparatus from the outside 
through a communication network 213. 

[0015] Thus, in this moving picture editing apparatus having the necessary program software stored in 
memory 202, when the image information of moving pictures being edited can be stored and managed in 
the information storage unit 206, and the moving pictures stored in the information storage unit 206 can be 
displayed on the display 208 in the form of icons of cut images and as icons of scene images. The icons of 
cut images and scene images form a tree structure, and the moving pictures can be edited according to the 
tree structure. 

[0016] The icons on the display 208 serves as an interface to the editing information of the cut images and 
scene images used in the moving picture edition and of the tree structure indicating the editing state. The 
editing operator uses for example a mouse of the input unit 209 to select this information on the display, 
thereby being able to instruct the moving picture editing apparatus. 

[0017] The operation of the apparatus for editing moving pictures will be described below. 

[0018] First, while viewing the images (not shown) indicated on the display 208, the user instructs the 
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apparatus by, for example, a mouse as the input unit 209 so that the image reproducer 205, for example, a 
video cassette recorder is operated to reproduce a video signal from a video tape. The reproduced signal is 
converted by the video interface 204 into image information of a necessary format, which is then stored in 
the information storage unit 206 through the bus 210. 

[0019] The converted image information of the necessary format from the video interface 204 is further 
supplied through the bus 210 to the image changing point detector 203. 

[0020] The scene changing point detector 203 compares the adjacent frame images of the input image 
information, and decides that when the image information as the difference between the frame images 
changes to exceed a predetermine amount the adjacent frame images respectively belong to different cut 
images. 

[0021] The cut images decided and separated at the changing point of images are respectively added with 
cut image numbers under the control of the CPU 201. 

[0022] The moving picture information formed of the frame numbers of cut images separated at the 
changing point, cut image numbers of cut images, file names of reduced images and file names of 
compressed moving pictures are stored in the information storage unit 206. 

[0023] In addition, the image information converted by the video interface 204 into a necessary format is 
supplied through the bus 210 to the compressor/expander 212. The compressed moving picture information 
from the compressor/expander 212 is stored in the information storage unit 206. 

[0024] Moreover, in order to confirm the whole moving picture formed of a plurality of frame images or the 
contents of the scene images or the edited program, the reduced images that can be fast read are also 
registered and stored in the information storage unit 206. 

[0025] The reduced images which can be fast read from the storage unit 206 are produced by decreasing 
the pixel number of the image thereby reducing the amount of data and the size of displayed image to 
match the size of icons displayed as representatives of cut images. 

[0026] The necessary scene images of the reduced moving picture, when the contents of scene images are 
desired to confirm, are caused to be displayed by, for example, a mouse of the input unit 209 on a display 
window 805 as shown in Fig. 8. 

[0027] Fig. 8 shows the screen of the display 208 displaying an editing window 801, a material area 803, an 
editing area 802 and an editing function button group 804. A plurality of icons of images representative of 
cut images are displayed on the material area 803 (although a single image representative of each of cut 
images is shown for simplicity, a number of frame images of each of cut images are actually displayed in a 
superimposed manner). The edited tree structure is shown on the editing area 802. 

[0028] If the changing points between cut images are not correctly detected, the changing points between 
cut images are indicated as being incorrect, so that the moving picture information in the information 
storage unit 206 can be reregistered under the control of the CPU 201. 

[0029] Then the operator orders to make editing of a moving picture by for example, the mouse of input unit 
209, hierarchical structure management information is read from the storage unit 206 or memory 202 and 
reregistered. 

[0030] A description will be made of the operation in the case where hierarchical structure management 
information according to the tree structure as for example shown in Fig. 1 1 is read. 

[0031] When the cut images #1 , #2 of the tree structure (Fig. 1 1 ) are ordered to be one scene image #1 . 
the information storage unit 206 is controlled by the CPU 201 to read a moving picture information file 901 
of data structure shown in Fig. 9 that correspond to cut images #1 , #2 and a hierarchical structure 
management information file 902 of data structure shown in Fig. 10. The scene image number is set to be a 
value "1", the addresses of cut image numbers #1, #2 as servants are set to be values "1", "2", and the 
displaying coordinates at which the icons of the two cut images are displayed are calculated. Then, the 
calculated values are set in cut image icon displaying coordinates of the hierarchical structure management 
information file, thus making re-registration. The same operations are performed for the scene image #2, 
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making re-registration into the hierarchical structure management information file. The management files 
shown in Figs. 9, 10 and 1 1 are generated or updated each time a video material is stored in the hard disk 
206. 

[0032] In addition, when the moving picture 1 formed of a plurality of scene images includes scene image 
#1 and scene image #2 as a one moving picture as shown in Fig. 11, a hierarchical structure management 
information file 903 shown in Fig. 10 is read from the information storage unit 206, the addresses of scene 
image numbers #1 , #2 as servants are set to be values "1", "2", respectively, and "moving picture #1" is set 
the number of master image of which the servant scene image numbers have values 1 , 2 set as above, 
thus making re-registration. 

[0033] Thus, the editing operations are repeated sequentially, so that the CPU 201 makes processing 
based on the icon-displayed coordinates, causing the tree structure shown in Fig. 11 to be displayed on the 
editing area 802 (see Fig. 8) of the screen of the display 208. 

[0034] The editing system for assembling the cut images and generating the hierarchical structure 
mentioned above is disclosed in the co-pending application Serial No. 08/970,202. The above-mentioned 
moving picture editing method and apparatus starts editing under the state in which editing image materials 
are previously stored in the storage unit. Therefore, it is necessary that the video input information 
transmitted from a video reproducing unit, or through a satellite channel or an exclusive line of a key station 
be previously stored in a hard disk in order to generate the tree structure after the recording in the 
information storage unit. The editing of images is an extremely delicate and important operation, and it 
takes a longer time than simply storing images in the information storage unit. Thus, if an extremely urgent 
program such as a news program cannot be previously edited, it will cause an important problem. 

[0035] In the above-mentioned moving picture editing method and apparatus, the tree structure can not be 
produced until the moving picture being edited is recorded on, for example, a video tape, reproduced by the 
video reproducer 205 and stored in the information storage unit 206 as necessary image information. Thus, 
since the moving picture is edited by necessary editing operations after the generation of the tree structure, 
the tree structure cannot be produced and the moving picture cannot be edited if the moving picture being 
edited is not stored in the information storage unit 206. 

[0036] In addition, let it be considered that the image information stored in the information storage unit 206, 
for example, the reduced moving picture file is erased. Since there is no longer reduced moving picture file 
in the moving picture information file, the moving information file cannot be read from the information 
storage unit 206 even if the operator tries to read the edited moving picture stored in the information 
storage unit 206 as a moving picture information file, and to change the moving picture edited state. 

[0037] Moreover, since the capacity of the information storage unit 206 is limited, the moving picture 
information file decided to be unnecessary is sometimes erased from the information storage unit. When 
the moving picture information file used for edition was erased by mistake, the edited information is lost and 
the moving picture must be reedited. Then only a file of the files registered in the moving picture information 
file is lost, it is necessary that the image material be again stored in the storage unit 206 and edited and 
that the tree structure be newly generated, and thus it takes a great deal of time. 

[0038] Embodiments of a moving picture editing method and apparatus according to the invention that can 
solve the above problems will be described below. 

[0039] Fig. 2 is a block diagram of a moving picture editing apparatus for executing a moving picture editing 
method according to the invention. The construction shown in Fig. 2 was previously described. The memory 
202 stores various control program software for CPU 201 and temporarily stores various edition control 
information used in the moving picture editing apparatus (a series of moving picture frame numbers, the 
frame numbers of detected cut image changing points, reduced images (icons)(size of. for example. 80x60 
pixels) that are produced from an inputted moving picture (size of, for example, 640x480 pixels) in order to 
be displayed on the display 208, cut image numbers sequentially attached to cut images, sequential 
numbers-indicating the order of a plurality of cut images and so on). 

[0040] The information storage unit 206 stores image information of a moving picture for edition reproduced 
from the image reproducer 205 and converted into a necessary format by the interface 204 and stores 
various editing control information, for example, image information files of icons that are temporarily stored 
in the memory 202 and used in the moving picture editing apparatus. This information storage unit 206 
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uses random-accessible recording media of, for example, hard disks, optical disks or magneto-optical 
disks. 

[0041] The elements connected to the bus 210 are controlled by access signals that are sent through the 
bus 210 from the CPU 201. 

[0042] The information storage unit 206 may be a remote file connected through the transmission network 
213. 

[0043] Although the necessary control program software are stored in the memory 202, they may be stored 
in another memory different from the memory 202. For example, they may be stored in a memory that is 
incorporated in the CPU 201. 

• 

[0044] In this moving picture editing apparatus, the memory 202 storing the program provided as will be 
described later, and the information storage unit 206 store and manage the image information of a moving 
picture being edited. The moving picture stored and managed in the information storage unit 206 is reduced 
to produce reduced cut images and scene images, and these reduced pictures are displayed as icons on 
the editing windows of the screen of the display 208. The cut images and scene images are edited, and the 
edited and registered cut images and scene images are displayed in a tree structure, thus leading to the 
generation of a program of a sequence of edited moving pictures. 

[0045] The above-given "cut image" is generally defined as a collection of frame images from the first frame 
to the last frame in one take shot by a television camera. The cut image is usually a moving picture. In the 
moving picture editing apparatus of the invention, the icons displayed on the screen of the display 208 
serve as a user interface having one-to-one correspondence with cut images. 

[0046] The "icon" is a reduced image to which a cut image is reduced in its size at a necessary reduction 
rate, and displayed as a representative of the cut image when all the moving picture comprised of a plurality 
of frame images, or scene images is fast read and displayed. 

[0047] The "scene image" is defined as a collection of a plurality of cut images each picked up for a certain 
object. A plurality of scene images are collected together to form one program for a particular subject. 

[0048] The moving picture edited as one program can be displayed and arranged on the display as a tree 
structure of information, that is, of a moving picture of sequential images (program) that is comprised of a 
plurality of cut images, scene images each being a collection of a plurality of cut images, and subjects each 
being a collection of a plurality of scene images. 

[0049] A description will be made of one example of the nioving picture edition in a moving picture editing 
method and apparatus according to the invention. 

[0050] In order to edit for a moving picture, first the image reproducer 205, for example, a video cassette 
recorder is operated to reproduce a medium, for example, a video tape having recorded thereon a moving 
picture that is desired to edit to form a program. The reproduced moving picture (signal) is converted by the 
video interface 204 into image information of a necessary format, and transmitted through the bus 210 to 
the information storage unit 206, where the moving picture (information) is stored. 

[0051] The image information converted into a necessary format by the video interface 204 is also supplied 
thrpugh the bus 210 to the cut image changing point detector 203. 

[0052] The cut image changing point detector 203 compares the adjacent frame images of the inputted 
image information, and decides that when the image information of the difference between the frame 
images changes to exceed a predetermined amount the adjacent frame images respectively belong, to 
different cut images. 

[0053] The moving picture information comprised of the frame numbers of the cut images detected in the 
changing points, and the names of fast displaying files and compressed image files is stored in the 
information storage unit 206. 

[0054] The moving picture stored in the information storage unit 206 is edited as above. The moving picture 
stored in the information storage unit 206 includes a necessary number of cut images as image materials 
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for making a program. 

[0055] Fig. 3 shows one example of the images that are displayed to edit on the display 208 of the moving 
picture editing apparatus by a moving picture editing method according to the invention. A blank icon 
generating button switch 305 and a coupling button switch 306 are provided in addition to the editing 
function button group 804 of the editing window 801 displayed on the screen of the display 208 shown in 
Fig. 8. Icons 310 SIMILAR 317 of cut images are also registered and displayed in a material area 303, but 
cut images have not been yet displayed edited without a tree structure in an editing area 302. 

[0056] The editing by use of a moving picture editing method and apparatus according to the invention will 
be described according to the procedure for editing operations. 

. [0057] The editing operation for a program of which the construction is always constant since it is 
repeatedly used will be mentioned first. For example, in a news program, a telop of program name is shown 
for five seconds at the start of the program, the scene of an announcer reading a news manuscript at a 
broadcasting studio for 20 seconds is shown after the end of the telop, a news picture of 30-second 
duration is shown after the end of the announcer's reading, and then the announcer's scene and the news 
scene including audio and text information are repeated three times, and finally a commercial (CM) picture 
continues for 15 seconds. 

[0058] In the editing window 301 shown in Fig. 3, a pointer 300 on the screen is moved to match the blank 
icon generation button switch 305 of the editing function button group 304 by use of, for example, a mouse 
of the input unit 209 of the apparatus. Then, the blank icon generation button 305 is clicked at the matched 
position by the mouse. 

[0059] When the blank icon generation button 305 is turned on, a window 401 for setting the length of cut 
image is displayed at a predetermined position in the editing window 301 as shown in Fig. 4. 

[0060] Since the telop of program name first runs for five seconds in a news program, a value 
corresponding to five seconds is inputted into a cut duration area 402 as "00:00:05:00" by use of a 
keyboard of the input unit 209. 

[0061] Then, when an OK button switch 403 is pushed by use of the mouse of the input unit 209 in the 
same way as described above, "1", "00:00:05:00". "NULL" and another "NULL" are respectively recorded in 
the addresses of the cut image number, cut image length, fast displaying file name and fast displaying 
image of the moving picture information file 901 shown in Fig. 9 that is stored in the information storage unit 
206. The "NULL" indicates that the corresponding image has not been registered yet or not present. 

[0062] Thereafter, the cut image length setting window 401 is closed, and an icon 310 is displayed in the 
material area shown in Fig. 3. This icon is a representative of the cut image having a data length of the 
value recorded in the moving picture information file 901. The edited images including the blank icons 
produced in accordance with above-mentioned editing process may be stored as some templates for 
moving picture edition into a memory, and the template may be read out from the memory to be used for 
other moving image editions in many times. 

[0063] Since NULL is recorded in the address of the fast displaying image of the moving picture information 
file 901, the icon 310 as a representative of the cut image is a blank icon that is displayed as a tentative 
indicator image for indicating that there is no corresponding cut image. An arbitrary cut image can be 
coupled to this blank cut image later. Further, a reduced picture of frame image having a constant signal 
level such as a black video signal may be used in place of the blank icon in the tree structure. 

[0064] Other programs can also be produced by repeating the same operations. As a result, blank icons 
31 1 SIMILAR 317 are displayed in the material area 303 as representatives of the cut images of other 
programs as shown in Fig. 3. 

[0065] When a CANCEL button 404 in the cut image length setting window 401 is pushed on, the cut length 
setting window 401 is closed without any recording in the moving picture information file 901. 

[0066] Then, the blank icons 310 SIMILAR 317 of cut images in the material area 303 are selected, and 
registered into the editing area 302. As a result, a tree structure can be obtained that is comprised of cut 
image icons 310 SIMILAR 317, and a scene image icon 318 as shown in the editing area 302 in Fig. 5. This 
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tree structure is generated in the seine way as described with reference to Figs. 8 to 1 1 . 

[0067] Pictures to be used for a news program are selected from the image material and registered in the 
cut image blanks, producing a news program as described below. 

[0068] The news images are reproduced from, for example, a video tape having collected images recorded 
thereon by the video reproducer 205. or supplied through a satellite communication receiving channel or an 
exclusive line between broadcasting stations, converted into a necessary format, and stored in the 
information storage unit 206. 

[0069] When the news images stored in the storage unit 206 are read by use of, for example, a mouse of 
the input unit 209 (although the images are read by the operation on the windows displayed on the display 
208 in the same way as described previously, the detailed description will not be omitted), the cut image 
changing point detector 203 detects and decides so that the images representative of the cut images 
separated at each changing point of the cut images are displayed as icons in the material area 303 as 
shown in Fig, 6 or 7. Fig. 6 shows icons of cut images orderly separated at each changing point of the cut 
images by the cut image changing point detector 203. Fig. 7 shows icons of cut images after rearrangement 
of those icons shown in Fig. 6. The image data of these icons are temporarily stored in the memory 202. 

[0070] Jhe detailed operations of detecting changing points of images, dividing by cutting, assembling the 
representative images of the cut images and generating a hierarchical structure are described in the co- 
pending U.S. Patent Application Serial No. 09/140,351 filed on Aug. 26, 1998 and titled "Method and 
Apparatus for Editing Moving Picture Image and Recording Medium for Storing Editing Program." The 
disclosures of this co-pending application are incorporated herein by reference. 

[0071] The cut images of a plurality of icons of news images displayed in the material area 303 are coupled 
to the previously provided blank icons 310 SIMILAR 317 displayed in a tree structure in the editing area 302 
as described below. A blank icon of a necessary cut image, for example, the blank icon 31 1 is connected to 
a cut image of necessary news images, for example, to the icon 322 by use of the mouse of the input unit 
209, That is, by moving the pointer to those icons to match, clicking the mouse button to instruct the 
connection, and then pushing on the couple button switch 306 of the edition function switch group 304 by 
the mouse of the input unit 209, the cut image is connected to the blank, so that the icon 322 of news 
images in the material area 303 is displayed as icons 31 1 in the editing area 302 as shown in Fig. 1. 

[0072] At this time the icon 322 of news images disappears from the material area 303 since the cut image 
has been coupled, or the file has moved to other area. Thus it can be perceived that the coupling has been 
made. 

[0073] Similarly, the cut image icon 321 of news images is coupled to the blank icon 313 of cut image in the 
editing area 302, and the cut image icon 323 of news Images to the blank icon 315 of cut image, so that the 
cut image icons 321 and 323 disappear from the material area 303. 

[0074] Although the news image icon 322 of "V" in the material area 303, for instance, is connected to the 
blank icon 31 1 , and disappears from the original place by the "movement" as described in the embodiment 
of Fig. 1 , it may be still left in the original area by "copy" after the coupling. 

[0075] If even one of the files registered in the moving picture information file 901 (see Fig. 9) of the storage 
unit 206 has been erased from the storage unit 206 and now does not exist, the image material will be 
required to be again stored in the storage unit 206, thus generating the tree structure in order for the erased 
file to be reregistered. The operation for avoiding this problem will be described below. 

[0076] The moving picture and file group of icons stored in the storage unit 206 are displayed as a list on 
the display 208 by use of. for example, the mouse of the input unit 209. 

[0077] When a file is erased from the storage unit 206 by the mouse of the input unit 209, the moving 
picture information file and hierarchical structure management information file in which the file name for 
Instructing to erase is stored are searched for, and "NULL" is written in the fast displaying file name and fast 
displaying image address of those files. 

[0078] Since "NULL" is written in the fast displaying image address of the moving picture information file so 
that blank icons are displayed, the images inserted in those blanks are required to change, but at this 
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moment when there are no image materials there is no need to again generate the tree structure and edit. 

[0079] One example of the program in the moving picture editing apparatus of the embodiment according to 
the invention will be described with reference to the flowchart of Fig. 12. First, at step 100, the operator 
inputs an instruction into the system of Fig. 2 through the input unit 209, so that the video material data for 
editing can be reproduced by the reproducer (VCR) 205 and stored in the magnetic storage unit 206. At 
step 101, the operator inputs the reduced image file name, moving picture file name and moving picture 
information file name into the system of Fig. 2 through the input unit 209. At step 102, each file name 
inputted at step 101 is recorded in the memory 202. At step 103, the video material is started to be 
recorded in the storage unit 206 from the reproducer 205. At step 104, setting of n=1 is made. At step 105, 
it is decided if an instruction has been issued through the input unit 209 to stop the storage unit 206 from 
storing the video material from the reproducer 205. If that instruction has been issued to stop the recording 
(YES), the program goes to step 106, where data of address h to address h+4*count+3 are read from the 
memory 202, and stored in the storage unit 206 as the moving picture information file (Fig. 9), and the 
program ends. 

[0080] If no instruction to stop the recording is issued at step 105 (NO), the program goes to step 107, 
where the video signal is reproduced frame by frame from the reproducer 205. At step 108, the video signal 
of one frame is supplied through the video interface 204 to the bus 210. At step 109. the one-frame video 
signal is supplied from the bus 210 to the scene changing point detector 203, CPU 201 and 
compressor/expander 212. At step 110, the CPU 201 generates a reduced irnage of 1/64 the original data 
amount by thinning out the one-frame video signal. At step 1 1 1, the reduced image is stored as a reduced 
image file in the magnetic storage unit 206. At step 112, the address of the reduced display file of the 
reduced image is stored in the memory 202. At step 113, the compressor/expander 212 makes data 
compression processing such as MPEG on the one-frame video signal. At step 1 14, the compressed video 
signal is stored as a moving picture file in the magnetic storage unit 206. At step 115, decision is made of if 
the one-frame image now being read is a changing point of scene. If it is a changing point of scene (YES), 
the program goes to step 116, the count is incremented as count=count+1 . The data "COUNT" is a number 
of counts utilized for writing a data into a predetermined address of the moving picture information file 901 
(Fig. 9) in step 122. When it is a changing point of scene, a new cut number is required, and thus at step 
117 the current cut number is incremented by 1. At step 118, the cut length is set for n-nl. The parameter 
n1 indicates the previous cut point. At step 119, a process of n1=n is made. At step 120, the reduced image 
file name and reduced image address are read from the memory 202. At step 121. the new cut number, the 
cut length, the reduced image file name and the reduced image are stored at the addresses of h+4*count, 
h+4*count+1, h+4*count+2 and h+4*count+3, respectively. At step 122, a process of n=n+1 is made, and 
then the program goes back to step 105, where the above operations are repeated for the next one-frame 
video signal. 

[0081] A supplementary explanation will be made of the procedure for generating the hierarchical structure 
management information file shown in Fig. 10. This management information file is generated when the 
hierarchical structure is produced by editing. 

[0082] When, for example, cuts #1 and #2 are combined into a scene, and ordered to be scene #1, data 
corresponding to the cuts #1, 2 are read as the moving picture information file of the construction shown in 
Fig. 9 from the magnetic storage unit 206. Value "1" is attached to the scene number, and values "1". "2" to 
the cut numbers #1 , #2. In addition, the display coordinates of the scene #1 are calculated from those of the 
two cuts and registered as the hierarchical structure management information file shown in Fig. 10 in the 
magnetic storage unit 206. The result is as shown in Fig. 1 3A. The scene #2 is also processed in the same 
way, and registered by adding to the hierarchical structure management information file. The result is as 
shown in Fig. 13B. In addition, as to a moving picture formed of a plurality of scenes, if a moving picture is 
formed of scene #1 and scene #2, the hierarchical structure management information file is read from the 
magnetic storage unit 206, values "1", "2" are attached to scene numbers #1. #2 as children, and a value 
"moving picture #1" is attached to the parent scene number with value "1" recorded. The result is the 
hierarchical structure shown in Fig. 11. 

[0083] In this invention, the cut image blanks for the tentative indicator image indicating that there are no 
particular corresponding cut images may be replaced by other icons, symbols, letters, figures or any 
combination of them. 

[0084] According to the invention, even if the moving picture to be edited is not stored in the information 
storage unit, a tree structure can be generated. In addition, since image edition can be made even if there 
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are no video materials, an urgent news program can be scheduled to broadcast by finishing the edition 
beforehand by the time when urgent video materials arrive. Thus the program edition can be performed 
more flexibly. 

[0085] In addition, the present invention can provide a moving picture editing method and apparatus in 
which even if the image information stored in the storage unit at the time of editing a moving picture is 
already erased, the moving picture can be edited after again storing the editing picture materials in the 
storage unit, without the editing operations for generating the tree structure. 
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Editing metliod and apparatus for moving pictures 

Claims of EP0969387 t Translate tWs te^^^^^ 

1. A moving picture editing method including operations for editing a moving picture fed from a video 
material source by working on edition working windows (301 SIMILAR 303) indicated on a display (208), 
said method comprising the steps of: 

dividing said video material into a plurality of cut images each formed of a plurality of frame images; 
displaying frame images (310 SIMILAR 31 7) representative of each of said cut images on the screen of 
said display; 

selecting said representative frame images, assembling a hierarchical structure (802) by a combination of a 
plurality of said frame images, and displaying said structure on said screen; 

selecting one or a plurality of said representative frame images as a scene image, and incorporating the 
frame image or images representative of said scene image into said hierarchical structure; and 
generating a tentative indicator image (310, 312, 314. 316, 317, 318) capable of representing an arbitrary 
video material according to an instruction, and incorporating said tentative indicator image in said 
hierarchical structure. . 



2. A method according to claim 1 , further comprising a step of making a selected cut image be associated 
with said tentative indicator image, replacing said tentative indicator image by frame images representative 

, of said selected cut image, displaying said frame images. 

3. A method according to claim 1 , wherein said tentative indicator image is a blank frame image. 

4. A method according to claim 1, wherein said tentative indicator image is at least one of icon, symbol, 
letter and figure capable of making a desired cut image be associated with said tentative indicator image. 

5. A method according to claim 2, wherein said selected cut image is one of said representative frame 
images displayed on the screen. 

6. A method according to claim 2, wherein said selected cut image is supplied through a communication 
channel. 

7. A method according to claim 1, further comprising a step of storing a template of a hierarchical structure 
including at least one tentative indicator image, and a step of relating an cut image to said tentative 
indicator image of the template. 

8. A moving picture editing apparatus for editing a moving picture fed from a video material source by 
working on edition working windows (301 SIMILAR 306) indicated on a display (208), said apparatus 
comprising: 

means (201 , 203) for dividing said video material into a plurality of cut images each formed of a plurality of 
frame images; 

means (201 ) for displaying frame images representative of each of said cut images on the screen of said 

display; 

means (201) for assembling a hierarchical structure formed by a combination of a plurality of said frame 
images In accordance with the selection of said representative frame images and displaying said structure 
on said screen; 

means (201) for selecting one or a plurality of said representative frame images as a scene image, and 
incorporating the frame image or images representative of said scene image into said hierarchical structure; 
and 

means (201, 305) for generating a tentative indicator image (310, 312, 314, 316. 317, 318) capable of 
representing an arbitrary video material, and incorporating said tentative indicator image in said hierarchical 
structure. 
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9. An apparatus according to claim 8, further comprising means for making a selected cut image be 
associated with said tentative indicator image, replacing said tentative indicator image by frame images 
representative of said selected cut image, and displaying said frame images. 

10. An apparatus according to claim 8, wherein said tentative indicator image is a blank frame image. 

1 1 . An apparatus according to claim 8, wherein said tentative indicator image is at least one of icon, 
symbol, letter and figure capable of making a desired cut image be associated with said tentative indicator 
image. 

12. An apparatus according to claim 9, wherein said selected cut image is one of said representative frame 
images displayed on the screen. 

13. An apparatus according to claim 9, wherein said selected cut image is supplied through a 
communication channel. 

14. An apparatus according to claim 8, wherein said video material source is a memory capable of random 
access. 

1 5. An apparatus according to claim 8, further comprising means for storing a template of a hierarchical 
structure including at least one tentative indicator image, and means for relating an cut image to said 
tentative indicator image of the template. 

16. A method of displaying images of editing operations of an editing apparatus for editing a moving picture 
fed from a video material source by working on editing working windows displayed on a display, said 
method comprising the steps of: 

displaying a first region for a video material and a second region for a hierarchical structure having a 
plurality of images of said video material in said first region on the screen of said display; 
displaying in said first region frame images representative of a cut image formed of a plurality of frame 
images; 

displaying in said second region said hierarchical structure formed of a plurality of said representative frame 
images; and 

displaying a tentative indicator image capable of representing an arbitrary video material at a specified 
position within said hierarchical structure. 



17. A moving picture editing apparatus according to claim 16, wherein said tentative indicator image is a 
blank frame image. 

18. A moving picture editing apparatus according to claim 16^, wherein said tentative indicator image is at 
least one of icon, symbol, letter and figure capable of making a desired cut image be associated with said 
tentative indicator image. 

19. A computer program product comprising a computer readable medium, having thereon: 

computer program means, when said program is loaded, to make the computer execute procedure to divide 
a video material into a plurality of cut images each formed of a plurality of frame images, display the 
representative frame images of each of said cut images on the screen of said display, select said 
representative frame images, assemble a hierarchical structure formed of a plurality of frame images, 
display said hierarchical structure, select one or a plurality of said representative frame images as a scene 
image, incorporate said frame images representative of said scene image in said hierarchical structure, 
generate a tentative indicator image capable of representing an arbitrary video material, and incorporate 
said tentative indicator image in said hierarchical structure, thereby editing a moving picture. 

20. A computer readable medium, having a program recorded thereon, where the program is to make the 
computer execute procedure to divide a video material into a plurality of cut images each formed of a 
plurality of frame images, display the representative frame images of each of said cut images on the screen 
of said display, select said representative frame images, assemble a hierarchical structure formed of a 
plurality of frame images, display said hierarchical structure, select one or a plurality of said representative 
frame images as a scene jmage, incorporate said frame images representative of said scene image in said 
hierarchical structure, generate a tentative indicator image capable of representing an arbitrary video 
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material, and incorporate said tentative indicator image in said hierarchical structure, thereby editing a 
moving picture. 
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